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This study explores how early childhood education programs are collecting and using 
data, how they would like to use data, how they could use the data that they have, 
and the challenges they face in these efforts. Administrators and teachers at seven 
preschools in a mid-sized city in the Northeast Region were interviewed about their data 
practices. Participating preschools used a variety of externally and internally developed 
systems to collect data on early learning outcomes, dosage (the amount of time children 
spend in early childhood education), and classroom quality. The preschools also provided 
data on early learning outcomes and dosage to parents, but some administrators and 
teachers had concerns about effective strategies for communicating findings from the 
data. The preschools reported collecting sufficient data and generally do not want to 
collect more data. 


This Stated Briefly report is a companion to the full report, Zweig, J., Irwin, C. W., Kook, J. F., 
& Cox, J. (2015). Data collection and use in early childhood education programs: Evidence from the 
Northeast Region (REL 2015-084). Washington, DC: U.S. Department of Education, Institute of 
Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional 
Educational Laboratory Northeast & Islands. Retrieved from http://ies.ed.gov/ncee/edlabs. 
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Why this study? 


Both federal and state policies increasingly require early childhood education practitioners to collect and use 
data within their programs (Stein, Freel, Hanson, Pacchiano, & Eiland-Williford, 2013). Research shows that 
early childhood education practitioners use data to monitor students’ learning and growth, examine progress 
toward state and district standards, become more knowledgeable about their own capacities, and develop 
plans for improvement (Crommey, 2000, and Earl & Katz, 2006, as cited in Datnow, Park, & Wohlstet- 
ter, 2007). Despite the increasing policy expectations for research-based practice and data-driven decision- 
making in early childhood education (Yazejian & Bryant, 2013), there is little research on the kinds of data 
that preschool educators collect and how they use data to enhance practice and inform decisionmaking. 

This study explores how early childhood education programs are collecting and using data, how they would 
like to use data, how they could use the data that they have, and the challenges they face in these efforts. 
It was conducted in collaboration with the Early Childhood Education Research Alliance at the Region- 
al Educational Laboratory Northeast & Islands. The alliance, which comprises state education leaders, 
prioritized a study examining the collection and use of data in preschools. Alliance members served as 
advisors on the study design and report. The audience for this study includes administrators of early child- 
hood education programs who are seeking to develop or enhance their data systems, policymakers who are 
considering policies to increase data-informed decisionmaking in preschools, and education leaders who 
are interested in advancing their data structures to answer more complex questions about early childhood 
education experiences and outcomes in K-12. 

What the study examined 


This study examined the data collected by early childhood education administrators and teachers in a mid- 
sized city in the Northeast Region; how they use the data they collected; and the challenges they face in 
collecting and using data. Based on previous research showing that dosage and classroom quality are posi- 
tively associated with early learning outcomes (see, for example, Burchinal, Kainz, & Cai, 2011; Burchinal 
et al., 2009; McCartney et al., 2010; NICHD Early Child Care Research Network, 2000; Peisner-Feinberg 
et al., 2001; Robin, Frede, & Barnett, 2006), this study focused on early learning outcomes, dosage, and 
classroom quality. Data on these topics have the potential to inform decisions about children, teachers, and 
early childhood education programs in general. 

How the study was conducted 


The study team conducted face-to-face interviews with administrators and teachers from a convenience 
sample of seven preschool programs in a mid-sized city in the Northeast Region. The participating pre- 
schools are state-licensed, center-based programs that accept children full-time, serve at least 40 pre- 
school-age children (defined by the state to be 33 months to 5 years old), and operate in the study city 
or a town within 10 miles of the study city. Interviewees reported on the availability and use of data on 
early learning outcomes, dosage, and classroom quality. The study team analyzed interview transcripts and 
determined the main themes by examining all responses aligned to research questions about the type of 
data collected and data use (box 1). The study team also analyzed child and classroom data from two 
of the preschools at which interviews were conducted in order to illustrate the potential advantages and 
challenges of using data; the results of that analysis are available in the full report (Zweig, Irwin, Kook, & 
Cox, 2015). 
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Box 1. Research questions 


The following research questions guided the study: 

• What data do administrators and teachers from a sample of preschools collect on early learning outcomes, 
dosage, and classroom quality? 

• How do these administrators and teachers use the data they collect? 

• How would these administrators and teachers like to use the data they collect? 

• What challenges do these administrators and teachers face in collecting and using data on early learning 
outcomes, dosage, and classroom quality that can inform policy or practice? 


What the study found 


Participating preschools used a variety of systems to collect data, but some had concerns about effective 
strategies for communicating findings from the data. Generally, preschools indicated that they considered 
the data they were collecting to be sufficient. 

The participating preschools used various systems, both externally and internally developed, for collecting data on 
early learning outcomes, dosage, and classroom quality 

Administrators and teachers at all seven preschools reported using ongoing, performance-based assess- 
ments of early learning outcomes. Four of the preschools used externally developed, commercially available 
assessment systems, including Teaching Strategies GOLD (Teaching Strategies, Inc., 2012) and the Work 
Sampling System (Meisels, Marsden, Jablon, Dorfman, & Dichtelmiller, 2012). The remaining three pre- 
schools relied on internally developed systems for collecting data on child outcomes, including anecdotal 
notes and work samples for children’s portfolios. 

The participating teachers were supported in a variety of ways in collecting data on early learning out- 
comes. Among teachers using Teaching Strategies GOLD, one reported that she had attended a formal 
training, one reported that only the administrator had attended but that the preschool planned to send all 
teachers for training in the future, and one reported completing an online training and meeting annually 
with the administrator and colleagues to review the system. The teacher using the Work Sampling System 
attended a formal course at a local teachers college. Finally, the three preschools that used internally devel- 
oped systems relied more heavily on teachers to devise their own systems for collecting data. One teacher 
talked with other teachers, and one regularly met with the administrator. 

Similarly, administrators at all seven preschools indicated that they regularly observed teachers to collect 
information about classroom quality, though their methods ranged in formality. Three preschools used 
the Early Childhood Environment Rating Scale — Revised (Flarms, Clifford, & Cryer, 1998), an externally 
developed instrument; two of these preschools also used a second instrument, the Arnett Caregiver Interac- 
tion Scale (Arnett, 1989) or the Classroom Assessment Scoring System (Pianta, La Paro, & Hamre, 2008). 
One administrator described an internally developed instrument that assessed four domains (classroom 
environment, interactions, planning and preparation, and personal qualities), and another administrator 
completed an open-ended form after each teacher observation that included notes and recommendations. 
The remaining preschools conducted informal observations without using an observation instrument. All 
teachers reported that they received feedback about their practice and felt supported by preschool staff — 
either directors or mentors. 
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Administrators at all the preschools reported that they collect daily attendance data — one measure of 
dosage — on all children. At six preschools, teachers collected attendance data using hard-copy binders 
or sign-in sheets that they submitted to their administrator to file as hard copies or enter into a digital 
database for storage purposes. At one preschool, teachers simultaneously collected and entered attendance 
data directly into a digital database. Preschool administrators indicated that they considered the collection 
process to be adequate for reporting to the state. 

Data on early learning outcomes and classroom quality were used to inform instruction and practice 

Administrators and teachers at all seven preschools reported using data on early learning outcomes to 
inform instruction and practice. One teacher explained that she used such data to determine what she 
needed “to work on.” Teachers reported that collecting data on early learning outcomes helped them track 
children’s progress and set appropriate learning goals. One teacher described an ongoing process of setting 
learning goals, using outcome data to determine whether those goals were reached, and then setting new 
goals as necessary. 

Similarly, seven administrators and six teachers indicated that they used classroom quality data to reflect 
on and improve their teaching practice. For example, one administrator stated that the purpose was “to 
reflect on what they’re doing in the classroom and make any adjustments to either their classroom man- 
agement, the way that they’re interacting with children, or what they’re actually presenting for activities to 
the children to help [them] develop.” One of the teachers described taking notes during meetings with the 
administrator to know what areas needed work and then addressing those areas the next day. 

All participating preschools provided data on early learning outcomes and dosage to parents, but some had concerns 
over effective strategies for communicating those findings 

Administrators at all seven preschools required early learning outcomes to be reported to parents, although 
the method of delivery varied. The three preschools that used internally developed systems relied more 
heavily on teachers to devise their own systems for presenting information to parents. The three preschools 
that used assessment systems provided teachers with quantitative output data for parents, but all adminis- 
trators reported that they encountered difficulty in knowing how best to communicate the data to various 
stakeholders. They described challenges with appropriately framing results for parents, grant-funding agen- 
cies, and the general public in ways that provided sufficient detail but were also easy to understand and not 
laden with jargon or complex figures. 

With regard to dosage, six administrators indicated that they reach out to parents of children who are 
frequently absent to report attendance data and provide information about state policies related to absen- 
teeism. Four preschool administrators reported that they were interested in linking attendance data to 
early learning outcomes. As one administrator explained, “If children are not here, they are not getting the 
instruction or the experiences.” That administrator wanted to be able to show parents the consequences of 
absenteeism. 

The participating preschools indicated that overall they considered the data they were collecting to be 
sufficient. Few teachers reported that they wanted to collect additional data on early learning outcomes, 
dosage, or classroom quality. One administrator stated a desire to collect additional data on children’s 
behavioral, social, and emotional outcomes, and one teacher who used an internally developed assessment 
system reported that she thought collecting data through an externally developed assessment system might 
be beneficial. All other interviewees indicated that they did not see a need to collect additional early learn- 
ing outcome data. Likewise, none of the teachers or administrators at the seven preschool programs wanted 
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to collect additional dosage data. When asked whether they would like to collect additional classroom 
quality data, one administrator and one teacher from two different preschools indicated that they would 
like to conduct more formal observations, but that there was not enough time to do this. 

Implications for policy and practice 


There are four main implications of this study: 

• Early childhood education programs may benefit from guidance on effective methods for presenting infor- 
mation to parents about children’s progress and about the importance of attending preschool. The par- 
ticipating preschools used data on early learning outcomes and attendance for outreach to parents; 
however, they expressed difficulty in knowing how best to present data to varying audiences. 

• Preschool administrators could benefit from more state guidance about successful data practices and 
structures. According to the Early Childhood Data Collaborative (2014), 32 states have designated 
an early childhood education data governance entity to guide the development and use of state-co- 
ordinated longitudinal early childhood education data systems. These governance structures could 
help guide preschools as they collect and use data. 

• Before instituting additional requirements for data collection, policymakers may want to weigh the ben- 
efits of additional data collection against the needs of practitioners and the time required to collect the 
information. Regardless of the instruments used or methods for collecting data on early learning 
outcomes, dosage, or classroom quality, the participating preschools generally did not want to 
collect additional data. Administrators and teachers considered their current data collection efforts 
to be sufficient. 

• Further research is needed to determine the most promising methods of practitioner data use that may 
lead to better outcomes for children. The participating preschools employed a variety of methods 
to collect and use data on early learning outcomes, dosage, and classroom quality. Although pre- 
schools could draw on existing research on data use in K-12 education, evidence on what data use 
methods in K-12 education help improve student achievement is inconclusive (Hamilton et al., 
2009). 
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